Effect of Document Representation on the Performance of Medical Document Classification

Text classification in the medical domain is a real world problem with wide applicability. This paper investigates extensively the effect of text representation approaches on the performance of medical document classification. To accomplish this objective, we evaluated seven different approaches to represent real word medical documents. The text representation approaches investigated in this paper are basic word representation (bag-of-words), key-phrases, collocation extracted from preprocessed text, collocation extracted from post-processed text, single-word-nouns, combination of single-word-noun and adjectives and combination of single-word-noun, adjective and verbs. A set of experiments was conducted to make comprehensive evaluation of the effects of these representation approaches using real world medical documents by measuring the classification performance. We measured classification performance with information retrieval metrics; precision, recall, F-measure and accuracy. Our experimental results show that bag-of-words representation outperforms all other representation approaches. In addition, careful use of selected features improve the classification performance.